The present invention relates to an apparatus for synchronized playback of audio-video (hereafter abbreviated as AV) signals, wherein compressed digital data of AV signals are decompressed into a video signal component and an audio signal component to be represented in synchronization with each other.
In the synchronized playback of AV signals, compressed digital data of AV signals read out from recording media such as a CD-ROM (Compact Disk Read Only Memory) or a hard disk device are decompressed before being represented through output devices such as a CRT monitor and a speaker. When the apparatus for the synchronized playback is provided with data processing ability sufficiently high for the data decompression of the AV signals, or when video signal size of the compressed digital data is comparatively small, the data decompression of both of the video signal component and the audio signal component can be performed within a short time, and so, audio data and video frames can be reproduced without interruption.
However, when the data processing ability of the apparatus is not sufficiently high compared to the video signal size of the compressed digital data, the apparatus can not reproduce both the audio signal component and the video signal component in real time with which they are recorded originally, taking too much time for the data decompression processes. In consequence, the video signal component may be represented as a slow-motion film and the audio signal component is reproduced intermittently.
The intermittent reproduction of the audio signal component gives sense of intense incongruity to the human hearing. Therefore, the audio signal component is usually processed to be reproduced continuously with a priority over the video signal component. The data decompression and displaying process of the video signal component is performed when the audio signal component is not processed. Here, adequate thin down of the picture frames (so-called frame omission) becomes necessary for synchronizing the presentation timings of the audio signal component and the video signal component so that the video signal component may not be delayed from the audio signal component.
As for the digital data compression, there is known an MPEG (Motion Picture Experts Group) standard, wherein the audio signal component and the video signal component separated of the AV signals are compressed independently into two data streams, an audio data stream and a video data stream.
FIG. 2A and FIG. 2B are schematic diagrams illustrating the video data stream 201 and the audio data stream 202, respectively. The video data stream 201 consists of a succession of GOP (Group Of Pictures) each composed of compressed data of picture frames, each denoted by a symbol I (Intra picture), B (Bidirectionlly predictive picture) or P (Predictive picture), while the audio data stream 202 consists of a succession of AAU (Audio Access Units) each including header information, error check bits, audio data and ancillary data. The video data stream 201 and the audio data stream 202 are multiplexed into a system stream at proper intervals, added with timing information called time stamps, so as to enable synchronized playback of the AV signals referring thereto.
As a prior art technique of the synchronized playback of the audio signal component and the video signal component from such multiplexed data, there is, for example, a Japanese patent application laid open as a Provisional Publication No. 140054/'96 (hereafter called the first prior art apparatus).
In the first prior art apparatus, vertical blanking intervals in a display signal are increased or decreased referring to the time stamps when the display signal is generated form data prepared in a video frame buffer so that representation timing difference between the audio signal component and the video signal component may not become more than a predetermined threshold value.
In another Japanese patent application laid open as a Provisional Publication No. 233269/'94 (hereafter called the second prior art apparatus), a video data unit of certain number of picture frames and corresponding amount of audio data unit are decompressed in parallel, unit by unit, and time difference between completion timings of decompression of the video data unit and the audio data unit is counted for every unit. Referring to the time difference, synchronization of the audio data and the video data is performed by increasing or decreasing display time of next picture frame, or by displaying the same picture frame repeatedly or omitting reproduction of following picture frames.
There are also Japanese parent applications laid open as Provisional Publication No. 165075/'94 and No. 140089/'96 (hereafter called the third prior apparatus). In the third prior apparatus, when a data error owing to data transmission error, for example, is detected in data receiving or data processing, the audio signal is muted in synchronization with the reproduction timing of the error data, for preventing jarring noises to be reproduced.
However, there are problems in these prior apparatus.
In the first prior apparatus, display timing of each picture frame is adjusted referring to difference of time stamp information of the audio data and video data. However, when the apparatus has not sufficient data processing ability, even if the vertical blanking interval is decreased, video data decompression delay itself cannot be recovered.
In the second prior apparatus, timing difference between decompression completion of the audio data unit and the video data unit is measured. Therefore, if there are some data errors in either or both the audio data and the video data, appropriate synchronization cannot be performed.
Furthermore, when the playback apparatus is implemented with software operating on an operating system with a low resolution software timer, there is another problem that correct synchronization between the audio signal component and the video signal component cannot be obtained because of the low resolution of the software timer, when the timing difference between decompression completion of the audio data unit and the video data unit is measured by the software timer. A fine resolution of a hardware timer, a CPU clock, for example, can be used, of course. However, use of the hardware timer makes the apparatus hardware-dependent, not applicable but to computers having a specific CPU clock hertz.
In the third prior apparatus, muting of the audio signal is performed when a data error is found. However, there is no description concerning the video signal reproduction or the synchronization of the audio signal component and the video signal component against the data error.